Multi-modal sensor fusion using a probabilistic aggregation scheme for people detection and tracking

Efficient and robust techniques for people detection and tracking are basic prerequisites when dealing with Human–Robot Interaction (HRI) in real-world scenarios. In this paper, we introduce a new approach for the integration of several sensor modalities and present a multi-modal, probability-based people detection and tracking system and its application using the different sensory systems of our mobile interaction robot HOROS. These include a laser range-finder, a sonar system, and a fisheye-based omni-directional camera. For each of these sensory systems, separate and specific Gaussian probability distributions are generated to model the belief in observing one or several persons. These probability distributions are further merged into a robot-centered map by means of a flexible probabilistic aggregation scheme based on Covariance Intersection (CI). The main advantages of this approach are the simple extensibility by the integration of further sensory channels, even with different update frequencies, and the usability in real-world HRI tasks. Finally, the first promising experimental results achieved for people detection and tracking in a real-world environment (our institute building) are presented. c © 2006 Elsevier B.V. All rights reserved.